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Abstract 

Functional constraints and bi-functional constraints are an important constraint class in 
Constraint Programming (CP) systems, in particular for Constraint Logic Programming 
(CLP) systems. CP systems with finite domain constraints usually employ CSP-based 
solvers which use local consistency, for example, arc consistency. We introduce a new ap- 
proach which is based instead on variable substitution. We obtain efficient algorithms for 
reducing systems involving functional and bi-functional constraints together with other 
non-functional constraints. It also solves globally any CSP where there exists a variable 
such that any other variable is reachable from it through a sequence of functional con- 
straints. Our experiments on random problems show that variable elimination can signif- 
icantly improve the efficiency of solving problems with functional constraints. 

To appear in Theory and Practice of Logic Programming (TPLP). 

KEYWORDS: constraint logic programming, constraint satisfaction problem, functional 
constraints, variable substitution, arc consistency 



1 Introduction 

Functional constraints are a common class of constraints occurring in Constraint 
Satisfaction Problem(s) (CSP) (Stallma n and Sussman 19771 |Van Hentenryck et al. 1 992 
Kirousis 1993). Roughly speaking, a constraint c(x,y) is functional if the value of 
variable y is some function of the value of variable x (see Definition [TJ. Func- 
tional constraints arise in two ways, they may occur quite naturally since one may 
have "functions" or "equations" in the constraint model. Functional constraints 
also occur systematically in in Constraint Programming (CP) when the system is 
a Constraint Logic Programming (CLP) system. 
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Functional constraints arise naturally in CLP in two ways. Firstly, the equa- 
tions which arise from matching the head of a rule with an atom in the body are 
functional constraints. Secondly, the basic (or primitive) constraints in a particu- 
lar instance of a CLP language will often include functional constraints. Consider, 
one of the most widely used and successful constraint domains for CLP, namely, 
finite domains which we will call CLP(FD). The basic constraints in a CLP(FD) 
system, for example, CHIP (Van Hentenryck et al. 1992), can express functional 
constraints. An example would be the finite domain constraint, 3X + 2Y = 10, 
with finite domain variables for X and Y0 Matching the head and body, gives rise 
to a number of equations, and in a FD system, the equations are functional con- 
straints. For example, when matching p(Z 2 + 1) with a rule on p(X) where both X 
and Z are finite domain variables, a functional constraint X = Z 2 + 1 is produced!! 
We remark that in logic programming, the equations are solved by unification but 
in the general setting, constraint solving over the particular domain is required. 
Recognizing and exploiting functional constraints can facilitate the development of 
more efficient constraint solvers for CLP systems. 

Most work on solving functional constraints follows the approach in CSP which 
is based on arc or path consistency ( |Van Hentenryck et al. 199"2{ IDavid 19"95l) . We 
remark that, in many papers, "functional constraints" are actually what we call bi- 
functional constraints (Definition [2]), a special case of functional constraints. In this 
paper, we propose a new method — variable substitution — to process functional 
constraints. The idea is that if a constraint is functional on a variable, this variable in 
another constraint can be substituted away using the functional constraint without 
losing any solution!! 

Given a variable, the variable elimination method substitutes this variable in all 
constraints involving it such that it is effectively "eliminated" from the problem. 
This idea is applied to reduce any problem containing non-functional constraints 
into a canonical form where some variables can be safely ignored when solving 
the problem. We design an efficient algorithm to reduce, in 0{ed 2 ) where e is the 
number of constraints and d the size of the largest domain of the variables, a general 
binary CSP containing functional constraints into a canonical form. This reduction 
simplifies the problem and makes the functional portion trivially solvable. When 
the functional constraints are also bi-functional, then the algorithm is linear in the 
size of the CSP. 

Many CLP systems with finite domains make use of constraint propagation al- 
gorithms such as arc consistency. Unlike arc consistency, our elimination method 
completely solves the functional portion of the problem, hence the functional con- 
straints are eliminated and their consequences are incorporated into the reduced 
problem. Our experiments show that the substitution based "global" treatment of 
functional constraints can significantly speed up propagation based solvers. 



1 Note that the examples which involve CLP use uppercase for variables as per the logic pro- 
gramming convention. In a more general context, we will use lowercase variables like x and y 
for variables. 

2 Notice that this is not a "linear" constraint in the arithmetic sense. 

3 A preliminary version of this paper appeared in l |Zhang et al. 2008^ . 
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In the rest of the paper, background on CSPs and functional constraints is given 
in Section [2j Variable substitution for binary functional constraints is introduced 
and studied in Section[3] Section[4]presents several results on algorithms for variable 
elimination in general CSPs containing functional constraints. Section [5] presents 
an experimental study on the effectiveness of the variable elimination algorithm 
and explains why functional elimination leads to a smaller problem with a reduced 
search space. In Section [6l we extend binary functional constraints to non-binary 
functional constraints and we discuss substitution for general problems where there 
are non-binary constraints which may be in extensional and intensional form. Re- 
lated work is discussed in Section [3 and the paper is concluded in Section O 

2 Preliminaries 

We begin with the basic concepts and notation used in this paper. 

A binary Constraint Satisfaction Problem (CSP) (N, D, C) consists of a finite 
set of variables TV = {v 1: ■ ■ ■ , v n }, a set of domains D = {Di, • • ■ , D n }, where Di is 
the domain of variable i , and a set of constraints each of which is a binary relation 
between two variables in N. 

A constraint between two variables i and j is denoted by cy. Symbols a and b 
possibly with subscript denote the values in a domain. A constraint is a set of 
allowed tuples. We assume testing whether a tuple belongs to a constraint takes 
constant time. For a G Di and b G Dj, we use either (a, b) G Cjj or Cij(a,b) to 
denote that values a and b satisfy the constraint c^ . For the problems of interest 
here, we require that for all a G Di and b G Dj, (a, b) G cy if and only if (b, a) G Cji. 
If there is no constraint on i and j, Cij denotes a universal relation, i.e., Di x Dj. 

A constraint graph G = (V,E) where V — N and E = {{i,j} | 3cy G C}. The 
constraint graph is usually used to describe the topological structure of a CSP. A 
solution of a constraint satisfaction problem is an assignment of a value to each 
variable such that the assignment satisfies all the constraints in the problem. A 
CSP is satisfiable if it has a solution. The solution space of a CSP is the set of all 
its solutions. Two CSPs are equivalent if and only if they have the same solution 
space. Throughout this paper, n represents the number of variables, d the size of 
the largest domain of the variables, and e the number of constraints in C . 

We need two operations on constraints in this paper. One is the intersection of 
two constraints (intersection of the sets of tuples) that constrain the same set of 
variables. The other operation is the composition, denoted by the symbol "o" of 
two constraints sharing a variable. The composition of two relations is: 

Cjk ° — {(a, c) | 3b G Dj, such that (a, 6) G Cjj A (b, c) G Cjk}- 

Composition is a basic operation in our variable substitution method. Composing 
c^ and Cjk leads to a new constraint on variables i and k. 

Example 1 

Consider constraints cy = {(ai, b\), (a 2 , 62), (02, b 3 )} and c ]k = {(&i, ci), (62, c 2 ), 
(63,c 2 )}. The composition of cy and Cjk is a constraint on i and k: Cik = {{a\,c\), 
(02,02)}- 
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Definition 1 

A constraint c y is functional on variable j if for any a 6 Di there exists at most 
one b G Dj such that Cy (a, b). cij is functional on variable i if Cj, is functional on 
i. Given a constraint Cy functional on variable j and a value a £ Di, we assume 
throughout the paper that in constant time we can find the value b S Dj , if there 
is one, such that (a, b) S Cy. 

A special case of functional constraints are equations. These are ubiquitous in 
CLP. A typical functional constraint in arithmetic is a binary linear equation like 
2x = 5 — 3y which is functional on x and on y. Functional constraints do not need 
to be linear. For example, a nonlinear equation x 2 — y 2 where x,y € 1..10 is also 
functional on both x and y. In scene labeling problems (jKirousis 1993[) . there are 
many functional constraints and other special constraints. 

When a constraint Cy is functional on variable j, for simplicity, we say Cy is 
functional by making use of the fact that the subscripts of Cy are an ordered pair. 
When Cy is functional on variable i, Cji is said to be functional. That cy is functional 
does not mean Cji is functional. In this paper, the definition of functional constraints 
is different from the one in ( |Zhang et al. 1999] |Van Hentenryck et al. 1992] ) where 
constraints are functional on each of its variables, leading to the following notion. 

Definition 2 

A constraint Cy is bi-functional if cy is functional on variable i and also on variable 
J- 

A bi-functional constraint is called bijective in (IDavid 1995|) . For functional con- 
straints, we have the following property on their composition and intersection: 1) 
If Cy and Cjk are functional on variables j and k respectively, their composition 
remains functional; and 2) The intersection of two functional constraints remains 
functional. 

Example 2 

The constraint Cy = {(a\, &i), (02, 61), (03, 62)} is functional, while the constraint 
Cy = {(ai, 63), (02, &i), (03, 62)} is both functional and bi-functional. An example 
of a non-functional constraint is Cy = {(ai, 61), (ai, 62), («2, (03, ^2)}- 

In the remainder of the paper, rather than writing Vi, we will simply refer to a 
variable by its subscript, i.e. i rather than Vi. 

3 Variable Substitution and Elimination Using Binary Functional 

Constraints 

We introduce the idea of variable substitution. Given a CSP (N, D,C), a constraint 
Cy 6 C that is functional on j, and a constraint Cjk in C, we can substitute j by i 
in Cjk by composing Cy and Cjk- If there is already a constraint Cik 6 C, the new 
constraint on i and k is simply the intersection of c^ and Cjk ° Cy. 
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Definition 3 

Consider a CSP (N,D,C), a constraint Cjj G C functional on j, and a constraint 
Cjk G C. To substitute j by i in Cjfc, using c^, is to get a new CSP where Cj-fc is 
replaced by c' ik — c ik n (cjk ° cy). The variable i is called the substitution variable. 

A fundamental property of variable substitution is that it preserves the solution 
space of the problem. 

Property 1 

Given a CSP (N,D 7 C), a constraint G C functional on j, and a constraint 
Cjk G C, the new problem obtained by substituting j by i in Cjfc is equivalent to 
(AT, AC). 

Proof 

Let the new problem after substituting j by i in Cjfc be (N, D, C) where C = 
(C - {c jk }) U {<4} and c' lk = c lk n (c jfe o c y ). 

Assume [a,\, a 2 , • • • , a„) is a solution of (N, D, C). We need to show that it satisfies 
C . The major difference between C and C is that C has new constraint c' ik . It 
is known that (ai,aj) G c^, (aj,ak) G Cjfc, and if there is Cik in C, (ai,ak) G c^. 
The fact that c' ik = (cjk ° cy) fl implies (aj, a^) G c' ik . Hence, c' ik is satisfied by 
(ai,a 2 , • • • ,a n ). 

Conversely, we need to show that any solution (a\, a 2 , • ■ • , a n ) of (N,D,C) is 
a solution of (N,D 7 C). Given the difference between C and C, it is sufficient 
to show the solution satisfies Cjk- We have (ai,aj) G Cjj and (ai 7 a k ) G c^ fc . Since 
c' ik = (cjk°Cij)CiCik, there must exist b G -Dj such that (otj, b) G c,j and (o, a^) G Cjfe. 
As Cij is functional, 6 has to be aj. Hence, aj and a k satisfy Cjk- □ 

Based on variable substitution, we can eliminate a variable from a problem so 
that no constraint will be on this variable (except the functional constraint used to 
substitute it). 

Definition 4 

Given a CSP (N,D,C) and a constraint G C functional on j, to eliminate j 
using c^ is to substitute j by i, using cy, in every constraint Cjk G C (except cji). 

We can also substitute j by z in Cj, to obtain c' u and then intersect c' u with the 
identity relation on Di, equivalent to a direct revision of the domain of i with respect 
to c^. This would make the algorithms presented in this paper more uniform, i.e., 
only operations on constraints are used. Since in most algorithms we want to make 
domain revision explicit, we choose not to substitute j by i in c^. 

Given a functional constraint c,j of a CSP (N,D 7 C) 7 let Cj be the set of all 
constraints involving j, except Cij. The elimination of j using results in a new 
problem (N, D, C) where 

C = (C - Cj) U {c' ik | c' ik = (c jk o Cij) n Cik, Cjk G C}. 

In the new problem, there is only one constraint c^ on j and thus j can be regarded 
as being "eliminated" . 
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Fig. 1. (a): A CSP with a functional constraint cy. (b): The new CSP after elimi- 
nating the variable j using cy . 

Example 3 

Consider a problem with three constraints whose constraint graph is shown in 
Figure[lja). Let cy be functional which this is indicated by the arrow in the diagram. 
The CSP after j has been eliminated using cy- is shown in Figure [ljb). In the new 
CSP, constraints Cjk and Cji are discarded, and new constraints c^ = cj^ o c« and 
ca = Cji oaj are added. Note that the other edges are not directed as the constraints 
Cjk, Cji, c^, Cu may not be functional. 

The variable elimination involves "several" substitutions and thus preserves the 
solution space of the original problem by Property [1] 

Corollary 1 

Given a CSP (N,D,C) and a functional constraint 6 C, the new problem 
(N,D,C) obtained by the elimination of variable j using cy is equivalent to 
(N,D,C). 

4 Elimination Algorithms for CSPs with Functional Constraints and 

Non-Functional Constraints 

We now extend variable elimination to general CSPs with functional and non- 
functional constraints. The idea of variable elimination (Definition |4] in Section [3|) 
can be used to reduce a CSP to the following canonical functional form. 

Definition 5 

A CSP (N, D, C) is in canonical functional form if for any constraint cy 6 C 
functional on j, the following conditions are satisfied: 1) if Cji is also functional on 
i(i.e., Cij is bi- functional), either i or j is not constrained by any other constraint 
in C; 2) otherwise, j is not constrained by any other constraint in C. 

As a trivial example, a CSP without any functional constraint is in canonical 
functional form. If a CSP contains some functional constraints, it is in canonical 
functional form intuitively if for any functional constraint cy, there is only one 
constraint on j. As an exception, the first condition in the definition implies that 
when c^ is bi- functional, one variable of might have several bi-functional 

constraints on it. 

In a canonical functional form CSP, the functional constraints form disjoint star 
graphs. A star graph is a tree where there exists a node, called the center, which we 
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call the free variable, such that there is an edge between this center node and every 
other node, which we call and eliminated variable. The constraint between the free 
variable and eliminated variable is that it is functional on the eliminated variable. 
In Figure HJa), assuming Cjk and Cji are functional on k and I respectively, then 
there would be directed edges (arrows) from j to k and j to /. After eliminating j, 
we get a star graph in Figure [T{b), since i will be the free variable at the center 
of the star graph, with free variables k and I. Notice that before eliminating j, 
Figure QJa) is a star graph, but the constraints are not in a canonical form. 

The constraint between a free variable i and an eliminated variable j is functional 
on j, but it may or may not be functional on i. In the special case that the star graph 
contains only two variables i and j and cy is bi-functional, one of the variables can 
be called a free variable while the other is called an eliminated variable. 

If a CSP is in canonical functional form, all functional constraints and the elim- 
inated variables can be ignored when we try to find a solution for this problem. 
Thus, to solve a CSP (N, D, C) in canonical functional form whose non-eliminated 
variables are NE, we only need to solve a smaller problem (NE, D' , C) where D' 
is the set of domains of the variables NE and C — {cij | cy £ C and i,j £ NE}. 

Proposition 1 

Consider a CSP P\ = (N, D,C) in a canonical functional form and a new CSP 
P2 = {NE, D' , C') formed by ignoring the eliminated variables in P\. For any free 
variable i £ N and any constraint cy £ C functional on j, assume any value of Di 
has a support in Dj and this support can be found in constant time. Any solution 
of P2 is extensible to a unique solution of P\ in 0(\N — NE\) time. Any solution 
of Pi can be obtained from a solution of Pi- 
Proof 

Let (ax, a2, • ■ ■ , o-\ne\) be a solution of (NE, D' , C). Consider any eliminated vari- 
able j £ N — NE. In C, there is only one constraint on j. Let it be cy where i 
must be a free variable. By the assumption of the proposition, the value of i in 
the solution has a unique support in j. This support will be assigned to j. In this 
way, a unique solution for (N,D,C) is obtained. The complexity of this extension 
is 0{\N - NE\). 

Let S be a solution of (N, D, C) and S' the portion of S restricted to the variables 
in NE. S' is a solution of (NE, D' , C) because C C C. S' can then be extended 
to S by using the functional constraints on the values of the free variables in S' to 
give unique values for the variables in N — NE. □ 

Any CSP with functional constraints can be transformed into canonical functional 
form by variable elimination using the algorithm in Figure [21 Given a constraint Cy 
functional on j, line 1 of the algorithm substitutes j by i in all constraints involving 
j. Note the arc consistency on Cik, for all neighbor k of i, is enforced by line 3. 

Theorem 1 

Given a CSP (N,D,C), Variable_Elimination transforms the problem into a 
canonical functional form in 0(n 2 d 2 ). 
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algorithm Variable_Elimination(inout (N, D, C), out consistent) { 
L <- N; 

while (There is dj G C functional on j where i,j G L and i j){ 
II Eliminate variable j, 

1. C <- {c' ik | c' ik <- (c jk o dj) n c ik ,c jk € C,k i} U (C - {c jk € C \ k i}); 

2. L^L- {j}; 

3. Revise the domain of i wrt du for every neighbour k of i; 
if (Di is empty) then { consistent false; return } 

} 

consistent <— true; 

} 

Fig. 2. A variable elimination algorithm to transform a CSP into a canonical func- 
tional form. 

Proof 

Assume Variable_Elimination transforms a CSP Pi = (N, D, C) into a new prob- 
lem P 2 = (N,D',C). We show that P 2 is in canonical functional form. For any 
constraint Cjj £ C functional on j, there are two cases. Case 1: j ^ L when the 
algorithm terminates. Since j £ L when the algorithm starts and line 2 is the only 
place where j can be removed from L, j must have been eliminated at certain step 
of the while loop. In line 1 (the component after "U"), all constraints on j (except 
Cij) are removed. That is is the unique constraint on j. Case 2: j £ L when the 
algorithm terminates. Since is functional on j, variable i is not in L when the 
algorithm terminates (otherwise, j will be substituted by line 1 at certain step of 
the while loop). Therefore, i is removed from L at certain step of the while loop, i is 
not substituted using Cki where k ^ j (otherwise C because of the elimination 
of i). This implies that i was substituted using Cji, and thus Cji is functional on i 
by the loop condition. Hence, is bi-functional, and i is not constrained by any 
other constraints (thanks to line 1). Therefore, cases 1 and 2 show that P 2 is in 
canonical functional form. 

Next, we prove the complexity of Variable_Elimination. The algorithm elim- 
inates any variable in at most once because once it is eliminated it is removed 
from L (line 2). Assume, before the algorithm, there is at most one constraint on 
any pair of variables (otherwise, we take the intersection of all constraints on the 
same pair of variables as the unique constraint). This property holds during the 
elimination process because in line 1, the intersection in the component before "U" 
guarantees that we have only one copy of constraint on any two variables. So, for 
each variable j and a constraint functional on j, there are at most n — 2 other 
constraints on j. The variable j in those constraints needs to be substituted (line 
!)• 

The complexity of the substitution j in each constraint is 0(d 2 ) which is the cost 
of the composition of a functional constraint and a general constraint. Recall that, 
for a functional constraint c^, given a value a G Di, we can find its support in 
Dj in constant time. To compose c^- with a general constraint Cjk, for each value 
a G Di, we find its support b G Dj (in constant time). If we take each constraint 
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Fig. 3. (a) The constraint graph of a CSP with functional constraints cy and Cjk- 
(b) A directed graph. 



as a matrix, the row of b of Cjk will be the row of Cjk ° Oij, which takes d steps. 
Therefore, the cost of computing Cjk ° Cij is 0(d 2 ). 

For n — 2 constraints, the elimination of j (line 1) takes 0(nd 2 ). There are at most 
n — 1 variables to eliminate and thus the worst case complexity of the algorithm is 
0(n 2 d 2 ). □ 

It is worth noting that the variable elimination algorithm is able to globally solve 
some CSPs containing non-functional constraints. 

Example 4 

Consider a simple example where there are three variables and k whose do- 
mains are {1,2,3} and the constraints are i = j, i = k + 1, and j ^= k. Note that 
although the constraints are listed in an equational form, the actual constraints are 
explicit and discrete, thus normal equational reasoning might not be applicable. By 
eliminating j using cy, Cjfc becomes {(2, 1), (3, 2)}, and the domain of i becomes 
{2, 3}. The non-functional constraint Cjk is gone. The problem is in canonical func- 
tional form. A solution can be obtained by letting i be 2 and consequently j = 2 
and k = 1. 

By carefully choosing an ordering of the variables to eliminate, a faster algorithm 
can be obtained. The intuition is that once a variable i is used to substitute for 
other variables, i itself should not be substituted by any other variable later. 

Example 5 

Consider a CSP with functional constraints cy and Cjk- Its constraint graph is 
shown in Figure [3(a) where a functional constraint is represented by an arrow. If 
we eliminate k and then j, we first get Cji t and cji 2 , and then get cu t and cu 2 . Note 
that k is first substituted by j and then later j is substituted by i. If we eliminate 
j and then fc, we first get Cik, and then get cu 1 and ca 2 . In this way, we reduce the 
number of compositions of constraints. 

Given a CSP P = (N,D,C), P F is used to denote its directed graph (V, E) 
where V — N and E = | cy € C and cy is functional on j}. Non- functional 

constraints in C do not appear in P F . A subgraph of a directed graph is strongly 
connected if for any two vertices of the subgraph, any one of them is reachable from 
the other. A strongly connected component of a directed graph is a maximum sub- 
graph that is strongly connected. To describe our algorithm we need the following 
notation. 
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Definition 6 

Given a directed graph (V, E) , a sequence of the nodes of V is a functional elimi- 
nation ordering if for any two nodes i and j, i before j in the sequence implies that 
there is a path from i and j. A functional elimination ordering of a CSP problem 
P is a functional elimination ordering of P F . 

The functional elimination ordering is used to overcome the redundant computa- 
tion shown in the example on Figure [3ja). Given a directed graph G, a functional 
elimination ordering can be found by: 1) finding all the strongly connected compo- 
nents of G; 2) modifying G by taking every component as one vertex with edges 
changed and/or added accordingly; 3) finding a topological ordering of the nodes 
in the new graph; and 4) replacing any vertex v in the ordering by any sequence of 
the vertices of the strongly connected component represented by v. 

To illustrate the process, consider the example in Figure |3th) which can be taken 
as P F for some CSP problem P. All strongly connected components are {ji, J2, J3}, 
denoted by c\, and {11,12,13}, denoted by ci- We construct the new graph by 
replacing the components by vertices: ({ci, C2}, {(ci, C2)}). We have the edge [c\, C2) 
because the two components are connected by (j'2, The topological ordering of 
the new graph is (ci, C2). Now we can replace c\ by any sequence of j's and C2 by 
any sequence of i's. For example, we can have a functional elimination ordering 

The algorithm Linear .Elimination in Figure [4] first finds a functional elimina- 
tion ordering O (line 1). The body of the while loop at line 4 is to process all the 
variables in O. Every variable i of O is processed as follows: i will be used to sub- 
stitute for all the variables reachable from i through constraints that are functional 
in C° and still exist in the current C . Those constraints are called qualified con- 
straints. Specifically, L initially holds the immediate reachable variables through 
qualified constraints (line 8). Line 9 is a loop to eliminate all variables reachable 
from i. The loop at line 11 is to eliminate j using i from the current C. In this 
loop, if a constraint Cjk is qualified (line 14), k is reachable from i through qualified 
constraints. Therefore, it is put into L (line 15). 

To illustrate the ideas underlying the algorithm, consider the example in Fig- 
ure[3jb). Now, we assume the edges in the graph are the only constraints in the prob- 
lem. Assume the algorithm finds the ordering given earlier: O = (73, j'2, ji, 12, 
Next, it starts from j'3. The qualified constraints leaving js are Cj 3 j 2 only. So, the 
immediate reachable variables through qualified constraints are L = {^2}- Take 
and delete j% from L. Substitute ji by % in constraints Cj 2 i 2 and Cj 2jl . As a result, 
constraints Cj 2 i 2 and Cj 2 j 1 are removed from C while c j3jl = Cj 3 j 1 D {cj 2 j 1 °Cj 3 j 2 ) and 
new constraint Cj 3 i 2 — Cj 2 i 2 °Cj 3 j 2 is introduced to C. One can verify that both Cj 2 j 1 
and Cj 2 i 2 are qualified. Hence, variables j\ and ii are reachable from j'3 and thus are 
put into L. Assume j\ is selected from L. Since there are no other constraints on 
ji, nothing is done. Variable ii is then selected from L. By eliminating 12 using J3, 
Ci 2 i x and Ci 2 i 3 are removed from C and Cj^ and Cj 3 i 3 are added to C. Constraint 
Ci 2 i x is qualified, and thus i\ is added to L. Note that Ci 2 i 3 is not qualified because 
it is not functional on i 3 in terms of the graph. We take out the only variable i\ in 
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algorithm Linear_Elimination(inout (N, D,C)) { 

1. Find a functional elimination ordering O of the problem; 

2. Let C° be C; any Cij in C° is denoted by c^; 

3. For each i € iV, it is marked as noi eliminated] 

4. while (O is not empty) { 

5. Take and delete the first variable i from O; 

6. if (i is not eliminated) { 

8. L {j | (i,j) € C and is functional}; 

9. while (L not empty) { 

Take and delete j from L; 
11. for any Cjfe G C — {cji} { // Substitute j by i in Cjt; 

c 'ik c jfc ° c a n c ifc; 
C^Cu{4}-fe}; 

14. if (c° fe is functional) then 

15. L^Lu{k}; 

} 

16. Mark j as eliminated; 
} II loop on L 

} 

} // loop on O 
} / / end of algorithm 

Fig. 4. A variable elimination algorithm of complexity 0(ed 2 ). 

L. After i\ is eliminated using j'3, Ci t i 3 is removed from C, and constraint Cj 3 i 3 is 
updated to be Cj 3 i 3 ("1 (c^^ oc^). Since c^^ is qualified, 13 is added to L. One can 
see that although i 3 was not reachable when i 2 was eliminated, it finally becomes 
reachable because of i\. All the variables in a strongly connected component are 
reachable from the variable under processing if one of them is reachable. Now, take 
is out of L, and nothing is done because there are no other constraints incident on 
it. Every variable except J3 is marked as eliminated (line 16), the while loop on O 
(line 4 and 6) terminates. 

Theorem 2 

Given a CSP problem, the worst case time complexity of 

Linear .Elimination is 0(ed 2 ) where e is the number of constraints and d the size 
of the maximum domain in the problem. 

Proof 

To find a functional elimination ordering involves the identification of strongly 
connected components and topological sorting. Each of the two operations takes 
linear time. Therefore, line 1 of the algorithm takes 0(n + e). 

The while loop of line 4 takes 0(ed 2 ). Assume that there is a unique identification 
number associated with each constraint in C. After some variable of a constraint is 
substituted, the constraint's identification number refers to the new constraint. 

For any identification number a, let its first associated constraint be cjk- Assum- 
ing j is substituted by some other variable i, we can show that i will be never be 
substituted later in the algorithm. By the algorithm, i is selected at line 5. Since 
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it is the first element of O now, all variables before i in the original functional 
ordering have been processed. Since i is not eliminated, it is not reachable from any 
variable before it (in terms of the original O) through qualified constraints (due to 
the loop of line 9). Hence, there are two cases: 1) there is no constraint c m i of C 
such that (P mi is functional on i, 2) there is at least one constraint c m i of C such 
that (P mi is functional on i. In the first case, our algorithm will never substitute i by 
any other variable. By definition of functional elimination ordering, case 2 implies 
that i belongs to a strongly connected component whose variables have not been 
eliminated yet. Since all variables in the component will be substituted by i, after 
the loop of line 9, there is no constraint c m i of C such that (P mi is functional on i. 
Hence, i will never be substituted again. 

In a similar fashion, if variable k is substituted by Z, Z will never be substituted 
later by the algorithm. 

So, there are at most two substitutions occurring to a. Each of these substitutions 
is a composition that involves a functional constraint. Hence, its complexity is 0(d 2 ) 
in the worst case as shown in the proof of Theorem [1] 

Since there is a unique identification number for each constraint, the total number 
of the unique identification numbers is e and thus the time taken by the while loop 
at line 4 is 0(ed 2 ). In summary, the worst case time complexity of the algorithm is 
0(ed 2 ). □ 

Before proving some properties of Linear_Elimination, we first define trivially 
functional constraints. 

Definition 7 

Given a problem P, let C° be the constraints before applying Linear .Elimination 
and C the constraints of the problem at any moment during the algorithm. A 
constraint cy of C is trivially functional if it is functional and satisfies the condition: 
c®j is functional or there is a path i\(= i),i2i ' • ' >*m(— j) in Co such that, Vfc £ 
l..m — 1, c? is functional on ih+i- 

Theorem 3 

Algorithm Linear Elimination transforms a CSP (N, D, C) into a canonical func- 
tional form if all newly produced functional constraints (due to substitution) are 
trivially functional. 

The proof of this result is straightforward and thus omitted here. 
Corollary 2 

For a CSP problem with non-functional constraints and bi-functional constraints, 
the worst case time complexity of algorithm Linear Elimination is linear to the 
problem size. 

This result follows the observation below. When the functional constraint involved 
in a substitution is bi-functional, the complexity of the composition is linear to the 
constraints involved. From the proof of Theorem[2j the complexity of the algorithm 
is linear to the size of all constraints, i.e., the problem size. 
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Corollary 3 

Consider a CSP with both functional and non-functional constraints. If there is a 
variable of the problem such that every variable of the CSP is reachable from it in 
P F , the satisfiability of the problem can be decided in 0(ed 2 ) using Linear .Elim- 
ination. 

For a problem with the property given in the corollary, its canonical functional 
form becomes a star graph. So, any value in the domain of the free variable is exten- 
sible to a solution if we add (arc) consistency enforcing during Linear .Elimination. 
The problem is not satisfiablc if a domain becomes empty during the elimination 
process. 

5 Experimental Results 

We experiment to investigate the effectiveness of variable elimination on problem 
solving. In our experiments, a problem is solved in two ways: (i) directly by a general 
solver; and (ii) variable elimination is applied before the solver. 

There are no publicly available benchmarks on functional constraints. We test 
the algorithms on random problems which are sufficiently hard so that we can 
investigate the effect of different numbers of functional constraints and the effect 
of constraint tightness. 

We generate random problems (n, d, e, nf, t) where n is the number of variables, 
d domain size, e the number of constraints, n/the number of functional constraints, 
and t the tightness of non-functional constraints. The tightness r is defined as 
the percentage of allowed tuples over d 2 . There are nf functional constraints and 
the rest of the binary constraints are non-functional. Each functional constraint 
is constructed to have d allowed tuples. In the context of random problems, the 
tightness factor of l/d due to the functional constraints is rather tight. When we 
increase nf, it can be the case that the search space is quickly reduced due to the 
effect of these very tight constraints. Therefore the "hardness" of the problems drops 
correspondingly when there is a significant increase of nf. As described below, we 
try to counter this effect by removing problems which be solved too easily from the 
benchmarks. 

In the experiments, we systematically test benchmark problems generated using 
the following parameters: n,d are 50, e varies from 100 to 710 with step size 122 
(710 is ~ 10% of the total number of possible constraints (1225)), nf varies from 
2 to 12, and t varies from 0.2 to 1.0 with step size 0.05. When nf is small (for 
example, 2), there are so many hard problems that we can only experiment with 
a small portion of the problems because it is computationally infeasible. When nf 
is large (for example, 12), even the most difficult problem instances from the set 
of instances becomes easy and only a small number of backtracks is needed. These 
instances can be solved too easily and thus are not very useful for benchmarking the 
elimination algorithm (i.e., we will not expect the elimination algorithm to bring 
any benefits to these instances). So, we do not include the cases with nf > 12. 
When nf = 12, the most difficult problems we found are with e = 710. Table Q] 
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tightness (t) 


0.2 - 0.65 0.7 


0.75 


0.8 


0.85 0.9 - 0.95 


#bt 


5.7 


22.9 


1023 


0.2 



Table 1 . Hardness of the problem versus tightness 



Table 2. The experimental results for random problems with n = d = 50. 



Elimination No Elimination 

cpu time (s) ^backtracks cpu time (s) ^backtracks 



344 


6 


0.60 


20042 


1.889e+06 


47781 


5.381e+06 


466 


7 


0.70 


9266 


9.362e+05 


35136 


3.955e+06 


588 


8 


0.75 


17922 


1.635e+06 


45464 


4.386e+06 


588 


9 


0.75 


10346 


5.679e+05 


21231 


1.605e+06 


710 


10 


0.80 


3039 


2.244e+05 


5771 


5.146e+05 


710 


11 


0.80 


481 


26522 


959.7 


71258 


588 


12 


0.75 


24 


682 


57.9 


2960 



shows the hardness of problems instances, with nf = 12 and e = 710, in terms of 
the number of backtracks #bt (average of 10 instances) needed. The hardness is 
measured using an arc consistency solver without using the elimination algorithm. 
When t is from 0.2 to 0.65, the problems are too easy (#bt is 0). For the most 
difficult case of t being 0.8, #bt is still rather small (around 1000). On the other 
hand, when nf is small, one can expect that the application of elimination may 
not make much difference. Therefore, we do not include the cases when nf is small 
either. 

Due to the observations above, we evaluate the algorithm only on non-trivial 
problem instances and where nf is not too tiny. For each nf (varying from 6 to 12), 
the results of the most difficult problem instances discovered in the exploration 
process above is shown in Table [2] The results were obtained on a DELL Pow- 
erEdge 1850 (two 3.6GHz Intel Xeon CPUs) with Linux. We implement both the 
elimination algorithm and a general solver in C++. The solver uses the standard 
backtracking algorithm armed with arc consistency enforcing algorithm after each 
variable assignment. During the search, the dom/deg heuristic is used to select a 
variable, and the value selection heuristic is in lexicographical order Q 

In Table [2] the cpu time is the total time of twenty problem instances for a given 
combination of e, nf and tightness, and the number of backtracks are their average. 
For the problem instances used in Table [2] the time to transform the instances into 
their canonical forms is negligible compared to the time needed for solving the 



4 The dom/dcg heuristic is a dynamic variable selection heuristic. 
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number of backtracks with elimination 
Fig. 5. The number of backtracks needed with and without elimination. 

instance. There are several reasons. First, the number of constraints involved in 
the elimination is relatively small compared to the total number of constraints 
in the problems. Second, the algorithm is as efficient as the optimal general arc 
consistency algorithm used in the solver. Thirdly, the elimination is applied only 
once to reduce the problem which can be done before the backtracking search, while 
the arc consistency algorithm needs to be called at every step during the search, 
i.e. roughly about the same as the number of backtracks. 

The results show that the variable elimination can significantly speed up the 
problem solving in terms of both cpu time and the number of backtracks. It reduces 
the number of backtracks by two to four times and also reduces the cpu time 
correspondingly. 

The statistics (cpu time and number of backtracks) used in Table [2] is for 20 
problem instances for each value of the selected parameters (each row in the table) . 
We notice that the hardness of these instances is not uniform, i.e., some instances 
are significantly harder than the others. To better visualize the performance of the 
algorithms, we replot the same data from Table [2] in Figure [5] Each data point in 
Figure [5j represents an instance whose ^-coordinate is the number of backtracks 
with elimination applied while its y-coordinate is that without elimination. Both 
axis use a log scale. 

The scatter plot in Figure [5] shows a similar performance improvement resulting 
from elimination. Note that all points above the line y = x indicate that not us- 
ing elimination requires more backtracks. We highlight the instances with nf = 7 
(the x symbol in the graph), which shows more extreme results as elimination can 
significantly speed up or slow down the problem solving. The slowing down is an 
interesting discovery of this experiment. An explanation is that variable elimina- 
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w/ elimination 1 X v " 

w/o elimination --X-- ^( 
I I I I I I I I I 

0.7 0.71 0.72 0.73 0.74 0.75 0.76 0.77 0.78 0.79 0.8 



tightness 

Fig. 6. The cpu time for instances with n = d — 50, nf = 8 and e = 588 and 
tightness varying from 0.70 to 0.80. 



tion changes the topology of the problem, which may affect the dynamic variable 
ordering heuristics of the general solver. It is well known that the performance of a 
constraint solver may vary significantly with a different variable ordering. 

To have a better idea on how the elimination algorithm performs, we look at 
instances with various hardness. We now zoom into the case of nf = 8 and e = 588 
— this has a large cpu time and is more tight from the the experiments in Table 
[2] We remark that our algorithm performs similarly in all these cases, so we just 
look at the details of a specific one with the results for all configurations where 
the tightness changes from 0.70 to 0.80 with a step of 0.01. The results are shown 
in Figure. |6] (cpu time) and Figured] (the number of backtracks). Again, the cpu 
time is the sum of the cost of 20 instances per parameter setting while the number 
of backtracks is their average. When the tightness is 0.79 and 0.80 the problem 
instances become very simple with less than 200 backtracks, we do not expect the 
elimination algorithm to improve the performance of the constraint solver although 
it reduces the number of backtracks. For most non trivial problems, elimination 
does help to improve the efficiency (both cpu time and the number of backtracks) 
significantly. When using 10 instances, we also observed that when the tightness is 
0.73, the elimination leads to a worse performance of the general problem solver in 
terms of both cpu time and the number of backtracks. 

As observed, when the number of functional constraints in the random problems 
increases, the problem instances become trivial (i.e., very few backtracks arc needed 
to solve them). This makes it hard to fully evaluate the elimination algorithm. To 
reduce the potential inconsistency caused by the tight functional constraints which 
makes the problems easy, we use identity functions (i.e., x — y, a special case 
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0.7 0.71 0.72 0.73 0.74 0.75 0.76 0.77 0.78 0.79 0.8 
tightness 

Fig. 7. The number of backtracks for instances with n — d = 50, nf = 8 and 
e = 588 and tightness varying from 0.70 to 0.80. 



Table 3. The experimental results for random problems with identity functions 
(the problem parameters are n = d = e = 100). 



Elimination No Elimination 

cpu time (s) ^backtracks cpu time (s) ^backtracks 



10 


0.04 


11.5 


779.6 


34.6 


1870.1 


20 


0.04 


1.4 


4.5 


9606.8 


131724.0 


30 


0.08 


1.1 


0.2 


40.7 


564.6 


40 


0.19 


1.1 





177.9 


1275.7 


50 


0.20 


0.7 





893.1 


64163.2 


60 


0.25 


0.6 





2.2 


1.4 



of functional constraint), instead of arbitrary functional constraints. We remark 
that we could also have used a permutation form of the identity function but that 
would have made the problem instance construction more complex. With identity 
functions, no inconsistency will result directly from functional constraints. The 
experimental results given in Table [3] show that we can create non trivial problems 
with much more functional constraints. 

For this set of instances, the elimination can speed up the problem solving by 
up to several orders of magnitude (in terms of both cpu time and the number of 
backtracks). An important observation is that when there is a significant amount 
of functional constraints, the number of backtracks needed after elimination can be 
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as small as 0. However, without elimination, the general problem solving may need 
a large number of backtracks (up to five orders of magnitude larger) . 

In summary, from our experiments, for non trivial random problems, elimination 
can improve the efficiency of a general constraint solver by several times to sev- 
eral orders of magnitude. We also observe that the elimination could make a solver 
slower possibly due to the change of the topological structure of the problems. How- 
ever, the slowdown only occurs rarely in our experiments. 



5.1 Search Space Reduction through Elimination 

The experiments show that the elimination could reduce the search space signifi- 
cantly. In fact, we can show that the elimination can help reduce the size of the 
search space (in terms of the current domains of the variables). 

Proposition 2 

Given a CSP problem P, let P' be the problem resulting from applying the elimina- 
tion algorithm to P. After enforcing arc consistency on P' and P, for each variable 
of P', its domain in P' is a subset of that in P. 

Proof 

Instead of proving the original proposition, we prove the following claim: given a 
CSP problem P = (V, D, C) and a constraint functional on j, let P' — (V — 
{j}, D' , C) be the problem resulting from the elimination of j. Any value, not from 
Dj, removed by enforcing arc consistency on P will be removed by enforcing arc 
consistency on P'. Let P 1 = (V, D 1 , C) and P' 1 = (V-{j}, D n ,C) be the result of 
enforcing arc consistency on P and P' respectively. Equivalently, we will show that 
there exists D'J C Dj such that P" = (V, D' 1 U {£>"}, C) (i.e., "plug" the domains 
of P n to P) is arc consistent. If this claim holds, the proposition holds by applying 
the claim repeatedly (as the elimination proceeds). 

Let the neighbors of j in P be i, ki, . . . , k m . For any constraint c/fc G C— {c,j, Cjk t , 
. . . , Cjk m }, cik G C Since cik is arc consistent in P' 1 , cik is arc consistent in P". 

We next show G C is arc consistent with respect to D'^ and Dj, i.e., for any 
value a G D'l, there is a support in Dj. Assume there is no support in Dj, by 
the definition of substitution, a has no support in any domain of k n (n G l..m), 
which contradicts that dk n G C is arc consistent with respect to D'^ and D'^ . 
Furthermore, let the support of a in Dj be b. We claim b has a support with 
respect to any constraint Cjk (k G {k\, . . . , k m }) in P' 1 . Otherwise, a has no sup- 
port with respect to Cik G C for some k G {fci, . . . , k m }, by the definition of 
substitution. It contradicts the fact that Cik is arc consistent in P' 1 . Let D'J = 
Dj — {b | b has no support with respect to Cji and D'^}. Clearly D'J is not empty 
(because is arc consistent in P' 1 ). It can be shown that Cji is arc consistent over 
domains D^ and D'J and is still arc consistent on these domains too. In other 
words, Cij is arc consistent in P" . 

Similarly, we can show that Cjk t , • • ■ , and Cjk m are arc consistent over P>k\ ( n £ 
ki..k m ) and D'J, and thus in P" . □ 
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Furthermore, after enforcing arc consistency on P and P' , for some variables of 
P', its domain in P' is a proper subset of that in P. Consider the following example: 
V = {x, y, z}, x,y,zE {1, 2}, and c xy = {(1, 1), (2, 2)}, c yz = {(1, 2), (2, 1)}, c M = 
{(1, 1), (2, 2)}. This problem is arc consistent and the domains of the variables are 
{1,2}. However, the problem P' resulting from variable elimination has an empty 
domain. 

Thus, we see that using the elimination algorithm together with arc consistency 
for the non-functional constraints leads to a "higher amount" of consistency. 

6 Beyond Binary Constraints 

Our presentation so far is based on binary constraints. To model real life problems, 
non-binary constraints are often useful. In this section, we discuss the potential 
extension of the work reported in this paper. The first subsection is to generalize 
substitution to non-binary constraints in extensional form, and the second proposes 
an approach to processing non-binary constraints in intcnsional form. 

6.1 Variable Elimination and Non-binary Functional Constraints 

In this section, we discuss the treatment of the generalization of binary functional 
constraints and substitution to non-binary constraints. We first generalize the func- 
tional property from binary constraints to non-binary constraints. Then, we show 
how a variable is substituted by a set of variables. 

A non-binary constraint is denoted by cs where S is the set of variables in the 
constraint. A linear equation x + y + z = 8 with finite domains for {x,y, z} is a 
non-binary constraint. We now define variable instantiations. An instantiation of 
a set of variables Y is an assignment of values to the variables in Y. It is usually 
denoted by a sequence. An instantiation is denoted by a character with a bar, for 
example, a. 

Definition 8 

A constraint cs is functional on j(e S) if for any instantiation a of S — {j}, there 
is at most one value of j such that this value and a satisfy c$- A constraint c$ is 
functional if it is functional on some variable j 6 S. 

Example 6 

Consider a constraint x + y + z = 8 with x,y,z € {1,2,3}. Let a = (1,1) and 
b = (2,3) be two instantiations of (x,y). For a, no value for z can be found to 
satisfy the constraint. For b, value 3 is the only value for z to satisfy the constraint. 
It can be verified that the constraint is functional on z, and similarly on x and on 

y- 

Example 7 

The constraint x 2 + y 2 + z = 8 with x,y,z e { — 1, —2, —3, 0,1,2, 3} is functional on 
z but not functional on x or y. 
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The idea of variable substitution is applicable to the functional non-binary con- 
straints defined above. However, we need more generalised operations to implement 
variable substitution. Below, we always take a constraint cs as a non-binary relation 
whose tuples are given explicitly. It is also helpful to recall that a relation is simply a 
set, and we can apply set operations like intersection to relations. For example, the 
constraint x + y + z = 8 with x,y,z £ {1,2,3} is taken as {(2,3,3), (3,2,3), (3,3,2)} 
where each tuple is an instantiation of variables (x, y, z). 

In the context of non-binary constraints, the composition of two constraints c$ 
and ct with respect to a variable i £ S D T, denoted by "oj," is defined below. 

cs Oj ct — {a | a is an instantiation of S U T — {i} and there exists a £ Di 
such that (a, a) satisfies both cs and ct-} 

If cs is functional on j and j is a variable of constraint ct, to substitute the 
variable j in ct in terms of cs is to replace ct by cs °j ct- 

Definition 9 

Consider a CSP (N, D, C) and two constraints cs and ct in C. Assume cs is 
functional on j £ SCiT. To substitute j in constraint ct using cs is to get a new CSP 
(N,D,C) where C = (C-{c T })U{c' SuT _ {j} } andc' SuT _ {j} = c SU T-{j}^(cs°jC T )- 

The variable substitution preserves the solution of a CSP. 

Property 2 

Given a CSP (N,D,C), a constraint eg £ C functional on j, and a constraint 
ct £ C where j £ T, the new problem obtained after j in ct is substituted using 
cs is equivalent to (N, D, C). 

Proof 

Let the new problem after j in ct is substituted be (N, D, C) where C — (C — 
{c T })U{ 

C SUT-{]}} anC ^ C SUT-{j} — c SUT-{j} H (Cs Oj Ct)- 

Assume a is a solution of (N, D, C). We shall show that a also satisfies C. 

Given a set of variables Y, ay will be used to denote the values in the solution a 
for the variables in Y. C differs from C in that it has the new constraint c'suT-{j}- 
It is known that as £ cs, 5t £ Ct, and if there is Cs<jT-{j} m C, a~suT-{j} satisfies 
CsuT-{j}- The fact that c' SljT _ {j} = (c s Oj c T ) n c SuT -{j} implies a SU T-{j} & 
c-suT-{j}(^ C')- Hence, dsuT-{j} 1S satisfied by a. 

Conversely, we need to show that any solution a of (N,D,C) is a solution of 
(N,D,C). Given the difference between C and C, it is only necessary to show a 
satisfies ct- To facilitate the following proof, we write as as (as~{j}, a,j), o~t as 
(aT-{j},aj). We have (as~{j}, af) £ cs and asuT-{j} € c 'suT-{j}- Assume, by con- 
tradiction, (aT-{j}, o,j) £ ct- Since asuT-{j} £ c sur-{j}' there must exist b £ Dj 
such that b ^ aj, (a~T-{j},b) satisfies ct, and (a~s-{j}, b) satisfies cs, contradicting 
that cs is functional on j. So, clt, that is (aT-{j},aj), satisfies ct- □ 

A CSP (N, D, C) with non- binary functional constraints can be reduced by vari- 
able substitution in a similar way as developed in this paper. In the non-binary case, 
we note that the complexity of the algorithm is more expensive due to the compo- 
sition operation (which is very close to the join operation in relational databases) . 
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6.2 Variable Elimination and Non-binary Constraints 

Non-binary constraints such as arithmetic or global constraints are common in CP 
systems. We discuss how variable elimination of functional constraints can be ap- 
plied to these constraints. Non-binary constraints are either in extensional (defined 
explicitly) or intensional (defined implicitly) form. To substitute a variable in an 
extensional non-binary constraint, we can follow the definition given the previous 
subsection. 

In most existing CP systems, for intentional constraints, there are usually par- 
ticular propagators with a specific algorithm associated with them. In this case, 
the approach using composition is not directly applicable simply because it has 
to interact with a constraint defined in terms of an arbitrary specific propaga- 
tion algorithm. We sketch below an approach which allows variable elimination 
to be employed with generic propagators. Assume we have a linear constraint c\\ 
ax + by + cz < d and a constraint c wy functional on y. To substitute y in c\, we 
simply modify c\ to be ax + bw + cz < d and mark w as a shadow variable (w 
needs special treatment by the propagator, which will be clear later). We call y the 
shadowed variable. Assume we also have c uw functional on w. To eliminate w, c\ is 
further changed to ax + bu + cz < d. Since w is a shadow variable, we generate a new 
constraint c uy using c uw and c wy in a standard way as discussed in this paper. Now 
u becomes the shadow variable while the shadowed variable is still y (variable w is 
gone). Suppose we need to make c\ arc consistent. First "synchronize the domains" 
of y and u using c uy , i.e., enforce arc consistency on c uy . (Note that due to elimina- 
tion, c wy and c uw are no longer involved in the constraint solving.) Next, we enforce 
arc consistency on c±. During the process, since u is a shadow variable, all domain 
operations are on y instead of u. After making c\ arc consistent, synchronize the 
domain of y and u again. (If the domain of u is changed, initiate constraint propaga- 
tion on constraints involving u.) This approach is rather generic: for any intensional 
constraints, synchronize the domains of the shadow variables and shadowed vari- 
ables, apply whatever propagation methods on the shadowed variables (and other 
non-shadow variables), synchronize the domains of shadow variables and shadowed 
variables again. In fact, the synchronization of the domains of the shadow and shad- 
owed variables (for example, u and y above) seems be readily implementable using 
the concept of views (|Schulte and Tack 2005j) . 



7 Related Work 

We now discuss variable substitution in the context of CLP followed by related work 
in variable substitution algorithms from other domains. Finally, the relationship 
to functional, bi-functional and other variable elimination approaches in the CSP 
literature. 
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7.1 CLP and Constraint Solving 

Logic Programming and CLP (IJaffar and Mahe~ 1994) systems often make use of 
variable substitution and elimination. The classic unification algorithm discussed 
below is a good example. 

A more complex example is CLP (TV) (jJaffar et al. 1992| which has constraints on 
finite trees and arithmetic. Variables in arithmetic constraints are substituted out 
using a parametric normal form which is applied during unification and also when 
solving arithmetic constraints. Our approach is compatible with such CLP solvers 
which reduce the constraint store to a normal form using variable substitution. We 
remark that any CLP language or system which has finite domain constraints will 
deal with bi-functional constraints simply because of the need to match an atom 
in the body with the head of a rule. The question is how powerful is the approach 
used. In this paper, we show that a variable substitution approach is more powerful 
than just simple finite domain propagation on equations. The consistency of the 
CSP is increased. Our experiments show that the time to solve the problem can be 
significantly smaller. 

7.2 Unification, Gaussian Elimination, and Elimination Algorithm for 

Functional Constraints 

The algorithm for unification of finite trees, the Gaussian elimination algorithm for 
linear constraints and our algorithm for functional constraints share the same key 
techniques: variable substitution and elimination. Such algorithms are also com- 
monly used in CLP systems. We illustrate this by examples. 

The first is the unification of finite trees or terms. Unifying two terms f(x, y, z) 
and f(y,z,g(x)), where x,y and z are variables, results in three term equations: 
x = y,y = z, z = g(x). These equations can be solved using a variable elimination 
method. We select a variable and eliminate it from the system by substitution. 
For example, we can select to eliminate x — substitute all x by y (using the first 
equation x — y), which results in y = z,z = (g(y)). This process continues until 
some constant symbols do not match in an equation, the left hand side variable 
appears in a sub-term on the right hand side (or vice versa), or no new equation 
can be produced and there is only one term equation left. 

Our second example is equation solving for arithmetic over the real numbers. A 
system of binary linear constraints on real numbers can be solved by the well known 
Gaussian elimination method. A major step is to select a variable and eliminate it 
using an equation. Specifically, to eliminate x using ax + by = c, we substitute all 
x in the system by (c — by) /a. 

Lastly, we look at the case of finite domain constraints considered in this paper. 
Given a binary CSP, when we have a variable x and a general constraint c yx func- 
tional on x, variable x will be eliminated by substituting x in the remainder of the 
constraints. The substitution here is achieved by (general) composition of relations. 
One can show that the substitution of x in Gaussian elimination produces an equa- 
tion (a constraint) which is the result of the composition of the involved equations 
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(constraints). In other words, given c yx : x = (c — by)/a and c xz : a\x + b\z = c\, the 
equation a\{c— by)/a+biz — c\ is equal to the composition of c yx and c xz . Therefore, 
the elimination method proposed in this paper can be regarded as a generalization 
of Gaussian elimination for linear equations to functional constraints defined over 
discrete domains. 

To see the further similarity among these algorithms, let us examine the impact of 
the variable elimination ordering on their efficiency. As shown in the earlier sections, 
if all the constraints are known a priori, we can find a good ordering to make 
the elimination algorithm more efficient. The same principle applies to unification 
algorithm. Consider the set of term equations x — f(a,a),y — f(x,x),z — f(y,y). 
Direct substitution using the ordering of x, y, z is more expensive than the ordering 
z, y, x. In fact, given a term equation, it can be unified in linear time by finding a 
good variable ordering QPaterson an d Wcgman 1978). 

In a CLP solver, the constraints are added to the constraint store dynamically. 
If the newly added constraint is a binary linear equation, it has been observed that 
one can improve the efficiency by choosing properly a variable to eliminate from 
the two involved in the equation ( |Burg et al. 1995] ) . For example, a brand new 
variable is preferred to an old one (a variable occurring previously in the constraint 
store). With the new variable, no substitution is necessary. For elimination using 
bi-functional constraints, the one involved in a lesser number of earlier constraints 
will be eliminated (Zh ang and Yap 2002[ ). In the case of unification, when there 
are several variables that can be eliminated, we choose the one that is involved in 
less number of constraints too (for example, (Escala-Imas and Ghallab 1988)). The 
variable selection idea, together with the disjoint set data structure and union-find 
algorithm, had led to almost linear algorithms in (Escala-Imas and Ghallab 1988; 
|Zhang and Yap 20020 . 



7.3 Functional Constraints and Variable Elimination in CSP 

We now discuss other work related to functional constraints from a CSP perspective. 

Bi-functional constraints, a special case of functional constraints, have been stud- 
ied in the context of arc consistency (AC) algorithms since Van Hentenryck et al. 
( |Van H entenry ck" et al. 1992[ ) proposed A worst case optimal AC algorithm with 



0(ed) was proposed in ( Van Hentenryck et al. 1992[ ). (In many of the papers, bi- 



functional constraints were called functional constraints). The special properties of 
bi-functional constraints were used to obtain the time complexity better than that 
of the optimal AC algorithms such as AC2001/3.1 (0(ed 2 )) (jBessiere et al. 20050 
for arbitrary binary constraints. A fast AC algorithm for a special class of increas- 
ing bi-functional constraints was also proposed in (|Liu 19950 . Here, our elimination 
algorithm solves the consistency of functional constraints and variable substitution 
incorporates the effect of the functional part of the problem into the rest of the 
non-functional constraints. Thus, it gives a higher level of consistency as it achieves 
global consistency for the functional constraints rather than local consistency like 
arc consistency. At the same time, it may simplify the remainder of the constraints, 
thus reducing the problem further. 
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A new type of consistency, label-arc consistency, was introduced in (lAffane and Bennaceur 1 996) 
and they showed that bi-functional constraints with limited extensions to other 
constraints can be (globally) solved, but no detailed analysis of their algorithms is 
given. In (Zha ng et al. 1999] ), we proposed a variable elimination method to solve 
bi-functional constraints in 0{ed). Bi-functional constraints also belong to the class 
of "Zero/One/ All" constraints which was shown to be one of the earliest classes 
of tractable constraints (Cooper et al. 1994 1. The subclass of "One" constraints in 
the "Zero/One/ All" class corresponds to bi-functional constraints. What was not 
realized in ( jCooper et al. 19 94; Zhang et al. 1999) was that because the concern 
was the tractability of the class of Zcro/One/All constraints, the importance of 
variable substitution and class of functional constraints was missed. We also point 
out that all the papers above deal with the special case of bi-functional constraints 
rather than functional constraints. 

David introduced pivot consistency for binary functional constraints in ([David 1995|) . 
Both pivot consistency and variable substitution are different ways of reducing a 
CSP into a special form. However, there are some important differences between 
pivot consistency and variable substitution in this paper. Firstly, the concept of 
pivot consistency, a special type of directional path consistency, is quite complex. 
It is defined in terms of a variable ordering, path (of length 2) consistency, and 
concepts in directed graphs. As we show in this paper, Variable substitution is a 
much simpler concept. It is intuitive and simple for binary CSPs, and it extends 
also simply and naturally to non-binary CSPs. Secondly, by the definition of pivot 
consistency, to make a CSP pivot consistent, there must be a certain functional 
constraint on each of the non-root variables. Variable substitution is more flexible. 
It can be applied whenever there is a functional constraint in a problem. Finally, 
to reduce a problem, the variable elimination algorithm takes 0(ed 2 ) while pivot 
consistency algorithm takes 0((n 2 —r 2 )d 2 ), where r is the number of root variables. 

Another related approach is bucket elimination ( |Dechter~T 999) . The idea in com- 
mon behind bucket elimination and variable substitution is to exclude the impact 
of a variable on the whole problem. The difference between them lies in the way 
variable elimination is performed. In each elimination step, substitution does not 
increase the arity of the constraints while bucket elimination could generate con- 
straints with higher arity (possibly with exponential space complexity). The former 
may generate more constraints than the latter, but it will not increase the total 
number of constraints in the problem. 

Another methodologically related work is bucket elimination (Dechter 1999). The 
common idea behind bucket elimination and variable substitution is to exclude the 
impact of a variable on the whole problem. However, they also differ in many as- 
pects. Bucket elimination deals with general constraints while variable substitution 
is applicable only to functional constraints. Bucket elimination assumes a variable 
ordering and eliminates the impact of a variable j on all relevant constraints that 
involve variables before j. In contrast, variable substitution can be used to elimi- 
nate the impact of a variable on any number of relevant constraints. The ways to 
eliminate a variable are different for the two methods. For example, consider the 
CSP shown in Figure [DJa) where 012,023,034, and C45 are functional. 
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Fig. 8. A CSP with variable ordering x\, X2, ■ • ■ , X5. 

Assume there is a variable ordering x±, xi, • ■ • , X5. The variables will be eliminated 
in the reverse of the variable ordering. When eliminating a variable, say variable X4, 
bucket elimination considers constraints only involving variables before (including) 
X4 and ignores other constraints. In this example, c 45 is ignored while constraints 
C14, C24, C3 4 are considered relevant. After eliminating X4, the new ternary constraint 
c {i,2.3} on xi,X2,X3 is added to the problem. 

In the variable substitution method, for variable x 4 and the constraint 034 func- 
tional on it, we can choose to substitute X4 in one or some of the constraints 041,042 
and C45, depending on a specific setting (for example, a static CSP or incremental 
CSP). If we choose to substitute X4 in all these constraints, new binary constraints 
231,232,035 are added and old constraints 041,042 and C45 are discarded. 

This example shows that in each elimination step, bucket elimination generates 
constraints with higher arity than variable substitution while the latter generates 
more constraints than the former. However, the variable substitution method will 
not increase the total number of constraints in the problem (because every time 
a new constraint is added, an old one is discarded). In the case of bi- functional 
constraints, it decreases the total number of constraints to n after all variables are 
eliminated. 



8 Conclusion 

We have introduced a variable substitution method to reduce a problem with both 
functional and non-functional constraints. Compared with the previous work on 
bi-functional and functional constraints, the new method is not only conceptually 
simple and intuitive but also reflects the fundamental property of functional con- 
straints. 

For a binary CSP with both functional and non-functional constraints, an al- 
gorithm is presented to transform it into a canonical functional form in 0(ed 2 ). 
This leads to a substantial simplification of the CSP with respect to the functional 
constraints. In some cases, as one of our results (Corollary 2) shows, the CSP is 
already solved. Otherwise, the canonical form can be solved by ignoring the elimi- 
nated variables. For example, this means that search only needs to solve a smaller 
problem than the one before variable substitution (or elimination). 

Our experiments show that variable elimination can significantly (in some cases 
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up to several orders of magnitude) improve the performance of a general solver in 
dealing with functional constraints. Our experiments also show some evidence that 
although rarely, the elimination could slow down the general solver in a non trivial 
way. 
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